Stereoscopic image capturing systems

ABSTRACT

A stereoscopic imager system, comprising: a sensor array comprising a first plurality of photosensors and a second plurality of photosensors spaced apart from the first plurality of photosensors by a gap, the first plurality of photosensors and the second plurality of photosensors being configured to detect ambient light in a scene; a moving component coupled to the sensor array and operable to move the sensor array between a first position and a second position within a full rotational image capturing cycle; and a system controller coupled to the sensor array and the moving component. The system controller can be configured to: move a field of view of a sensor array by instructing the moving component to capture a first image of an object in the scene with the first plurality of photosensors from a first perspective at the first position, and to capture a second image of the scene of the object in the scene with the second plurality of photosensors from a second perspective at the second position; and calculate, based on the first image and the second image, a distance to the object using an optical baseline defined by the gap.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.63/016,118, filed Apr. 27, 2020, which is incorporated by referenceherein.

BACKGROUND

Image capturing systems capture images of a scene by sensing light. Thelight is typically sensed by an image sensor, such as a charge-coupleddevice (CCD) or a complementary metal-oxide semiconductor (CMOS) device,that can convert sensed light into electrons. The electrons can then beread and interpreted to construct the captured image. The image capturedby the image sensor often does not provide a perception of depth forconstructing a stereoscopic image of the scene, nor does it enable thecalculation of distance to objects in the scene.

SUMMARY

Some embodiments of the disclosure pertain to stereoscopic imagecapturing systems that can capture depth information with a sensorarray. A stereoscopic image capturing system can be configured to moveits sensor array, e.g., by rotating the array about an axis transverseto its rows, so that a given location in space can be successivelyimaged by at least two photosensors that are spaced apart from oneanother. The distance between the two photosensors can be used as anoptical baseline for calculating depth information to the given locationin the field, thereby enabling the stereoscopic image capturing systemto not only construct images with a perception of depth from a 2D sensorarray, but also to determine depth information to augment rangingaccuracy of depth sensors.

Some embodiments pertain to a stereoscopic image capturing system thatincludes both ranging and imaging photosensors. The stereoscopic imagersystem can include: a sensor array comprising: a plurality of rangingphotosensors that detect light emitted from an emitter array once it hasreflected off of an object in a scene; a first plurality of imagingphotosensors positioned at a first side of the ranging photosensors; anda second plurality of imaging photosensors positioned at a second sideof the ranging photosensors opposite from the first side. The firstplurality of imaging photosensors and the second plurality of imagingphotosensors can detect ambient light in the scene and can be spacedapart by a gap. The system can further include a moving componentcoupled to the sensor array and operable to move the sensor arraybetween a first position and a second position within a full rotationalimage capturing cycle; and a system controller coupled to the sensorarray and the moving component. The system controller can be configuredto: determine a first distance to an object in the scene using theplurality of ranging photosensors by way of time-of-flight calculations;capture a first image of the scene at the first position with the firstplurality of imaging photosensors and a second image of the scene at thesecond position with the second plurality of imaging photosensors; andcalculate a second distance to the object based on the first image andthe second image and an optical baseline determined by the gap.

In some implementations, embodiments can include one or more of thefollowing features. The plurality of ranging photosensors can beorganized in a diagonally staggered arrangement. The first and secondpluralities of imaging photosensors can each be organized in rectangulararrangements. At least some of the first plurality of imagingphotosensors and at least some of the plurality of second imagingphotosensors can be positioned along the same horizontal line. Themoving component can be an electric motor that rotates the sensor arrayaround a center axis. The moving component can be a micro-electricalmechanical system (MEMS) device that reflects light to move the field ofview. The system controller can be further configured to calculate afinal distance to the object based on the first distance and the seconddistance.

According to some embodiments, a stereoscopic imager system includes: asensor array comprising a first plurality of photosensors and a secondplurality of photosensors spaced apart from the first plurality ofphotosensors by a gap, the first plurality of photosensors and thesecond plurality of photosensors being configured to detect ambientlight in a scene; a moving component coupled to the sensor array andoperable to move the sensor array between a first position and a secondposition within a full rotational image capturing cycle; and a systemcontroller coupled to the sensor array and the moving component. Thesystem controller can be configured to: move a field of view of a sensorarray by instructing the moving component to capture a first image of anobject in the scene with the first plurality of photosensors from afirst perspective at the first position, and to capture a second imageof the scene of the object in the scene with the second plurality ofphotosensors from a second perspective at the second position; andcalculate, based on the first image and the second image, a distance tothe object using an optical baseline defined by the gap.

In some embodiments a method of distance measurement is provided wherethe method includes: moving a field of view of a sensor array includinga first imaging photosensor and a second imaging photosensor spacedapart from the first imaging photosensor by a gap; capturing a firstimage of an object in a scene with the first imaging photosensor from afirst perspective at a first instance of time as the field of viewmoves; capturing a second image of the scene of the object in the scenewith the second imaging photosensor from a second perspective at asecond instance of time as the field of view moves; and calculating,based on the first image and the second image, a first distance to theobject using an optical baseline defined by the gap.

In various implementations, the method can include one or more of thefollowing. Moving the field of view can include rotating the sensorarray around a center axis. Moving the field of view can includereflecting light to move the field of view while the sensor array isstationary. The first array of imaging photosensors and the second arrayof imaging photosensors can each be two-dimensional arrays of imagingphotosensors. The sensor array can be formed of a two-dimensional arrayof imaging photosensors, and the first array of imaging photosensors andthe second array of imaging photosensors can each be a subset of thetwo-dimensional array of imaging photosensors. The method can furtherinclude comparing shared features of the object captured in the firstimage and the second image, and using the results from the comparison tocalculate the first distance to the object. The method can furtherinclude measuring a second distance to the object using a rangingphotosensor, and determining a final distance to the object based on thefirst distance and the second distance. And, the ranging photosensor canbe in a two-dimensional array of ranging photosensors and the first andsecond arrays of imaging photosensors can be located on opposing sidesof the array of ranging photosensors.

A better understanding of the nature and advantages of embodiments ofthe present disclosure may be gained with reference to the followingdetailed description and the accompanying drawings. It is to beunderstood, however, that each of the figures is provided for thepurpose of illustration only and is not intended as a definition of thelimits of the scope of the disclosure. Also, as a general rule, andunless it is evident to the contrary from the description, whereelements in different figures use identical reference numbers, theelements are generally either identical or at least similar in functionor purpose.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example passive stereoscopic imagersystem, according to some embodiments of the present disclosure.

FIG. 2 is a simplified top-down illustration of an example lightdetection system configured to perform stereoscopic imaging, accordingto some embodiments of the present disclosure

FIGS. 3A-3B are simplified top-down illustrations of a rotating lightdetection system during different instances of time of an imagecapturing sequence, according to some embodiments of the presentdisclosure.

FIG. 3C is a top-down illustration of light detection system during thedifferent instances of time shown in FIGS. 3A-3B superimposed over oneanother, according to some embodiments of the present disclosure.

FIG. 4 is a simplified illustration of an example of two offset imagescaptured by first and second arrays of photosensors at first and secondinstances of time, respectively, according to some embodiments of thepresent disclosure.

FIG. 5A is a simplified illustration of an example sensor arrayconfigured as two separate linear arrays of photosensors that are spacedapart from one another for performing stereoscopic imaging, according tosome embodiments of the present disclosure.

FIG. 5B is a simplified illustration of an example sensor arrayconfigured as two separate m×n arrays of photosensors that are spacedapart from one another for performing stereoscopic imaging, according tosome embodiments of the present disclosure.

FIG. 5C is a simplified illustration of an example sensor arrayconfigured as a single m×n array of photosensors having two subsets ofphotosensors that are spaced apart from one another for performingstereoscopic imaging, according to some embodiments of the presentdisclosure

FIG. 6 is a block diagram of an example active stereoscopic system,according to some embodiments of the present disclosure.

FIG. 7 is a simplified diagram of an example light detection system,according to some embodiments of the present disclosure

FIG. 8 is a simplified illustration of an example sensor array for anactive stereoscopic imager system, according to some embodiments of thepresent disclosure.

FIG. 9 is a block diagram of an example method of operating astereoscopic imager system to perform stereoscopic imaging using lightdetections systems, according to some embodiments of the presentdisclosure.

FIG. 10A is a simplified cross-sectional view diagram of part of a lightdetection system where there is no cross-talk between channels.

FIG. 10B is a simplified cross-sectional view diagram of part of a lightdetection system where there is cross-talk between channels.

FIG. 11 is a simplified cross-sectional diagram of an examplemicro-optic receiver channel structure, according to some embodiments ofthe present disclosure.

FIGS. 12A-12B are simple illustrations of example implementations ofstereoscopic imager systems, according to some embodiments of thepresent disclosure.

DETAILED DESCRIPTION

Stereo imaging is a technique that creates or enhances the perception ofdepth using two offset two-dimensional (2D) images of the same field ofview. Data generated from stereo imaging, along with a known distancebetween the sensors that captured the 2D image (i.e., “opticalbaseline”), can be used to determine distances to locations within thefield of view. Imager systems that perform stereo imaging to captureperception of depth and to determine distances are herein referred to as“stereoscopic imager systems”.

Some embodiments of the present disclosure pertain to stereoscopicimager systems that have an image sensor whose field of view can bemoved/rotated across a scene. The sensor array can include one or morearrays of photosensors, or one or more subsets of photosensors in asingle, two-dimensional array. In some instances, the sensor arrayincludes a first array of photosensors spaced apart from a second arrayof photosensors, where the first and second array of photosensors areconfigured to capture images of a scene from two different perspectivesfor stereo imaging purposes. In order to allow the sensor array tocapture images from two different perspectives, the viewing direction ofeach photosensor in the first array of photosensors can intersect theviewing direction of corresponding photosensors in the second array ofphotosensors, as will be discussed further herein with respect to FIG.2. That way, when the viewing direction of the first and second arraysof photosensors are moved across the scene, the field of view capturedby the first array of photosensors at a first position can overlap withthe field of view of the second array of photosensors at a secondposition after the field of view of the sensor array has moved from thefirst position to the second position. As an example, the sensor arraycan be rotatable around an axis so that when the sensor array rotatesaround the axis, the first and second arrays of photosensors can captureimages of the field from two different perspectives at differentinstances of time to allow calculation of distance to objects in thescene, as will be discussed further herein with respect to FIGS. 3A-3C.

In some embodiments a stereoscopic imager system can be a passive systemthat does not actively illuminate a scene and instead detects ambientlight in the scene reflected off of one or more objects in the scene. Apassive stereoscopic imager system can include a light sensing modulefor receiving ambient light in the field. The light sensing module caninclude one or more bulk receiver optics, a micro-optic receiver system,and a system controller for operating the light sensing module. Themicro-optic receiver system can include one or more micro-optic receiverlayers and one or more photosensors that can measure received light, aswill be discussed further herein with respect to FIG. 11.

In some alternative embodiments, the stereoscopic imager system can bean active system that can emit light into a field and then detect theemitted light after it has reflected off surfaces of an object in thefield. An active stereoscopic imager system can include a lighttransmission module in addition to a light sensing module, and beconfigured as a light ranging device. The light transmission module caninclude a transmitter layer that is composed of an array of individualemitters (e.g., vertical-cavity surface-emitting lasers (VCSELs)) whereeach emitter can be paired with a corresponding micro-optic receiverchannel and corresponding photosensor of a sensor array in the lightsensing module, or it can be a uniform illuminator that spreads lightevenly across the scene with no specific pairing between individualemitters and receiver channels. In some instances, the lighttransmission module can include a micro-optic transmitter channel arrayto enhance light outputted from the array of emitters. During operation,light (e.g., laser pulses) outputted by the array of emitters passesthrough the micro-optic transmitter channel array and enters a bulktransmitter optic having a large numerical aperture to better capturelight from the micro-optic transmitter channel array. The light thenexits the bulk transmitter optic and illuminates a plurality of spots ata distant field.

In such active stereoscopic imager systems, the 2D sensor array caninclude an array of ranging photosensors for receiving light, e.g.,narrowband laser light, emitted by the emitter array for calculatingdistances via time-of-flight measurements and the like, and an array ofimaging photosensors for receiving ambient, red-green-blue (RGB) lightfor stereo imaging purposes, as will be discussed further herein withrespect to FIGS. 7 and 8. By using both ranging photosenors and imagingphotosensors, ranging accuracy can be augmented using distance datacalculated from stereo images captured by the imaging photosensors.

To better understand the function and configuration of passive andactive stereoscopic imager systems according to embodiments of thedisclosure, each will be discussed in detail herein.

I. Passive Stereoscopic Imager System

FIG. 1 is a block diagram of an example passive stereoscopic imagersystem 100, according to some embodiments of the present disclosure.Passive stereoscopic imager system 100 can include a system controller104 and a light sensing module 106. Imaging data can be generated bypassive stereoscopic imager system 100 by receiving light existing in ascene in which passive stereoscopic imager system 100 is positioned. Thereceived light can be light that exists naturally in the field, i.e.,ambient light, as opposed to light emitted from a transmitter withinsystem 100.

Light sensing module 106 can include a sensor array 108, which can be,e.g., a two-dimensional array of photosensors. Each photosensor can be aCCD, CMOS, or any other suitable sensor for detecting ambient light.Light sensing module 106 includes an optical sensing system 110, whichwhen taken together with sensor array 108 can form a light detectionsystem 112. In some embodiments, optical sensing system 110 can includea bulk receiver optic 114 and optical components 116, such as anaperture layer, a collimating lens layer and an optical filter, that canbe combined with sensor array 108 to form an array of micro-opticreceiver channels where each micro-optic receiver channel measures lightthat corresponds to an image pixel in a distinct field of view of thesurrounding field in which system 100 is positioned. Further details ofvarious embodiments of micro-optic receiver channels according to thepresent disclosure are discussed in detail in conjunction with FIG. 11herein.

A bulk imaging optic as defined herein can be one or more opticalsurfaces, possibly including multiple lens elements, that have clearapertures greater than one millimeter and that is positioned to receivelight projected from, or focus received light on, a micro-optictransmitter/receiver layer. A bulk imaging optic that projects lightreceived from an optical emitter, such as a micro-optic transmitterlayer, is sometimes referred to herein as a bulk transmitter optic or asan output bulk imaging optic. A bulk optic layer that focuses lightreceived from a field onto an optical detector, such as a micro-opticreceiver layer, is sometimes referred to herein as a bulk receiver opticor as an input bulk imaging optic. An input, image-space telecentricbulk imaging optic allows the system to measure narrowband lightuniformly over a wide field-of-view (FOV).

In some embodiments, sensor array 108 of light sensing module 106 isfabricated as part of a monolithic device on a single substrate (using,e.g., CMOS technology) that includes both an array of photosensors, aprocessor 118, and a memory 120 for signal processing the measured lightfrom the individual photosensors (or groups of photosensors) in thearray. The monolithic structure including sensor array 108, processor118, and memory 120 can be fabricated as a dedicated ASIC. In someembodiments, optical components 116 can also be a part of the monolithicstructure in which sensor array 108, processor 118, and memory 120 are apart. In such instances, optical components 116 can be formed, e.g.,bonded (non-reversibly) with epoxy, on the ASIC so that it becomes partof the monolithic structure. As mentioned above, processor 118 (e.g., adigital signal processor (DSP), microcontroller, field programmable gatearray (FPGA), and the like) and memory 120 (e.g., SRAM) can perform thesignal processing. As an example of signal processing, for eachphotosensor or grouping of photosensors, memory 120 of light sensingmodule 106 can accumulate detected photons over time, and these detectedphotons can be used to recreate an image of the field.

In some embodiments, the output from processor 118 is sent to systemcontroller 104 for further processing, e.g., the data can be encoded byone or more encoders of the system controller 104 and then sent as datapackets to user interface 115. System controller 104 can be realized inmultiple ways including, e.g., by using a programmable logic device suchan FPGA, as an ASIC or part of an ASIC, using a processor 122 withmemory 124, and some combination of the above. According to someembodiments of the present disclosure, processor 122 can receivestereoscopic images from light sensing module 106 and use them togenerate images with a perception of depth as well as to calculatedistances to objects in a scene, as will be discussed further herein.System controller 104 can cooperate with a stationary base controller oroperate independently of the base controller (via pre-programedinstructions) to control light sensing module 106 by sending commandsthat include start and stop light detection and adjust photodetectorparameters. In some embodiments, system controller 104 has one or morewired interfaces or connectors for exchanging data with light sensingmodule 106. In other embodiments, system controller 104 communicateswith light sensing module 106 over a wireless interconnect such as anoptical communication link.

According to some embodiments of the present disclosure, passivestereoscopic imager system 100 can also include a moving component 126coupled to system controller 104 and light sensing module 106. Movingcomponent 126 can be controlled by system controller 104 to move thefield of view of sensor array 108. In some instances, movement of sensorarray 108 can be achieved by physically moving light sensing module 106or by redirecting the field of view of light sensing module 106 by wayof light reflection. As an example, moving component 126 can be a motor(e.g., an electric motor) that rotates light sensing module 106 aroundan axis perpendicular to the rows of photosensors in sensory array 108,as will be discussed further herein with respect to FIGS. 4A-4C.Alternatively, moving component 126 can be a light redirectingcomponent, such as a microelectromechanical system (MEMS) device thatcan be modulated to reflect the light of a two-dimensional array oflight emitters in different directions to capture the two offset 2Dimages of the field. The two offset 2D images can then be used togenerate a stereoscopic image of the scene for calculating distancemeasurements and/or for generating images with a perception of depth.

A. Light Detection System for Performing Stereoscopic Imaging

FIG. 2 is a simplified top-down illustration of an example lightdetection system 200 configured to perform stereoscopic imaging,according to some embodiments of the present disclosure. Light detectionsystem 200 can include a sensor array 202 mounted on a heat sink 204 fordissipating heat generated by sensor array 202 during operation. Sensorarray 202 can be positioned behind a bulk receiver optic 206 so thatlight propagates through bulk receiver optic 206 before exposing onsensor array 202. Light detection system 200 can be enclosed within anoptically transparent housing 208 to protect light detection system 200from the environment.

In some embodiments, sensor array 202 can include a plurality ofphotosensors that are arranged to enable capture of two offset 2D imagesof a scene. For instance, sensor array 202 can include more than onearray of photosensors that are spaced apart from one another, or asingle array of photosensors with subsets of photosensors that arespaced apart from one another, so that when the field of view of thephotosensors is moved, two offset 2D images can be captured. As anexample, sensor array 202 can include a first array of photosensors 210and a second array of photosensors 212 separated by a gap 214. First andsecond arrays of photosensors 210 and 212 can each be an m×n array ofphotosensors, and the dimension of the field of views of both first andsecond arrays of photosensors 210 and 212 can be the same.

In some instances, first and second arrays of photosensors 210 and 212can capture light from chief rays propagating along differentdirections. For example, first array of photosensors 210 can capturelight from chief rays 220 propagating along a first direction, andsecond array of photosensors 212 can capture light from chief rays 222propagating along a second direction, where the first and seconddirections intersect one another. That way, when the field of view ofsensor array 202 is moved, e.g., by physical rotation 216 around acenter axis 218, the field of view of first array of photosensors 210can overlap with the field of view of the second array of photosensors212 at different instances of time during rotation of light detectionsystem 200, and each array can capture the same image of the scene, butfrom different perspectives for stereo imaging and distance calculationpurposes using gap 214 as an optical baseline. A better understanding ofthis operation can be understood herein with reference to FIGS. 3A-3C.

FIGS. 3A-3B are simplified top-down illustrations of rotating lightdetection system 200 during different instances of time of an imagecapturing sequence, and FIG. 3C is a top-down illustration of lightdetection system 200 during the different instances of time superimposedover one another, according to some embodiments of the presentdisclosure. Specifically, FIG. 3A shows light detection system 200 at afirst instance of time, and FIG. 3B shows light detection system 200 ata second instance of time after the first instance of time but beforelight detection system 200 has made a full 360° rotation around itscenter axis, e.g., center axis 218 in FIG. 3. During operation, lightdetection system 200 can continuously rotate 360° while its photosensorsare activating at a high frequency to capture images of itssurroundings, e.g., its scene, for stereoscopic imaging and distancecalculation.

As shown in FIG. 3A, light detection system 200 may be positioned asshown in FIG. 3A at the first instance of time while it is rotatingaround center axis 218. At that time, first array of photosensors 210 ofsensor array 202 can be positioned so that its field of view 300captures an image 301 of a tree 302 from a first perspective via chiefrays 220 passing through bulk receiver optic 206. Meanwhile, secondarray of photosensors 212 can be positioned so that its field of view304 captures an image of a region to the right of tree 302 via chiefrays 222 passing through bulk receiver optic 206.

As light detection system 200 continues to rotate, it may be positionedas shown in FIG. 3B at the second instance of time but before a completerotation has been made since the position of light detection system 200at the first instance of time. At the second instance of time, secondarray of photosensors 212 can be positioned so that its field of view304 captures an image 303 of tree 302 from a second perspective viachief rays 222 passing through bulk receiver optic 206. Meanwhile, firstarray of photosensors 210 can be positioned so that its field of view300 captures an image of a region to the left of tree 302 via chief rays220 passing through bulk receiver optic 206. The second perspective canbe different from the first perspective so that image 301 and image 303form a pair of offset images of tree 302.

In some embodiments, field of view 300 of first array of photosensors210 has the same dimension as the field of view 304 of second array ofphotosensors 212. In such embodiments, the size and shape of first arrayof photosensors 210 can be equal to the size and shape of second arrayof photosensors 212. For instance, first and second array ofphotosensors 210 and 212 can be an m×n array of photosensors of acertain pitch that is suitable for capturing an image of tree 302 in thescene. Thus, the field of view of second array of photosensors 212 atthe second instance of time can completely overlap the field of view offirst array of photosensors 210 at the first instance of time.Accordingly, image 301 of tree 302 captured by first array ofphotosensors 210 at the first instance of time and image 303 of tree 302captured by second array of photosensors 212 at the second instance oftime can be images of the same field of view of the scene but from twodifferent perspectives. This pairing of images at different fields ofview can be continuously repeated as light detection system 200continuously rotates around its center axis to capture images of itssurrounding scene.

The difference in perspective can be achieved by the relativepositioning of first and second arrays of photosensors 210 and 212 astree 302 at which images 301 and 303 are captured. As shown in FIG. 3C,the position at which first array of photosensors 210 captures image 301of tree 302 is offset from the position at which second array ofphotosensors 212 captures image 303 of tree 302 by a distance 308defined by gap 214. Distance 308 is therefore an optical baseline thatcan be used to triangulate the distances of the surface of tree 302 fromlight detection system 200 and for depth perception purposes. As can beappreciated with reference to FIG. 3C, distance 308 can be greater thangap 214 due to the fact that the axis of rotation, e.g., center axis 218in FIGS. 3 and 3A-3B, is not positioned at the center of sensor array202, but rather at a position below sensor array 202. Thus, in theembodiment shown in FIGS. 3A-3C, distance 308 is defined by 214 but isnot equal to it. This is not intended to be limiting, however, becauseother embodiments can have the axis of rotation be positioned at thecenter of sensor array 202 where the optical baseline is equal to thegap between the arrays of photosensors.

As can be appreciated from the discussion above, the two images capturedby first array of photosensors 210 at the first instance of time andsecond array of photosensors 212 at the second instance of time can beimages of the same field of view of the scene but from two differentperspectives and thus form two offset images suitable for stereoscopicimaging and distance calculation purposes. Distances can be calculatedby comparing the two images and measuring a separation distance betweenthe same features of objects evident in the two images. An example ofsuch a comparison is discussed herein with respect to FIG. 5.

FIG. 4 is a simplified illustration of the two offset images 301 and 303captured by first and second arrays of photosensors 210 and 212 at thefirst and second instances of time, respectively, according to someembodiments of the present disclosure. Given the difference inperspectives, image 301 captured by first array of photosensors 210 atthe first instance of time can show tree 302 differently than image 303captured by second array of photosensors 212 at the second instance oftime. For instance, tree 302 can appear to be laterally condensed inimage 303. With the two images 301 and 303 superimposed over one another(as opposed to being pictured beside one another as shown in FIG. 4 forclarity), distances between the same features of objects in the images,such as distances 400, 402, and 404 between the same branch, trunk, androot, respectively, can be used for triangulation purposes.

It is to be appreciated that rotational movement of light detectionsystem 200 shown in FIGS. 3A-3C is merely one way to move the field ofview of light detection system 200, and that embodiments are notintended to be so limited. Other types of mechanisms for moving thefield of view to enable stereoscopic imaging are envisioned herein. Forinstance, a movable mirror, such as a rotating mirror/MEMS device can beused to reflect the chief rays captured by the photosensors of thesensor array so that the photosensors can capture two offset images ofthe scene as discussed herein.

B. Sensor Array for Passive Stereoscopic Imaging

As aforementioned herein, a sensor array for enabling stereoscopicimaging can include photosensors that are spaced apart from one anotherso that when the field of view of the sensor array is moved, two offset2D images can be captured. According to some embodiments, such a sensorarray can be constructed various ways. For example, a sensor array caninclude more than one array of photosensors that are spaced apart fromone another, or a sensor array can include a single array ofphotosensors with subsets of photosensors that are spaced apart from oneanother, as will be discussed further herein with respect to FIGS.5A-5C.

FIGS. 5A-5C are simplified illustrations of example sensor arraysconfigured in various ways. Specifically, FIG. 5A is a simplifiedillustration of an example sensor array 500 configured as two separatelinear arrays of photosensors that are spaced apart from one another,FIG. 5B is a simplified illustration of an example sensor array 501configured as two separate m×n arrays of photosensors that are spacedapart from one another, and FIG. 5C is a simplified illustration of anexample sensor array 503 configured as a single m×n array ofphotosensors having two subsets of photosensors that are spaced apartfrom one another, for performing stereoscopic imaging, according to someembodiments of the present disclosure.

As shown in FIG. 5A, sensor array 500 can include a first array ofphotosensors 502 and a second array of photosensors 504 separated by agap 506. First and second arrays of photosensors 502 and 504 can each bea linear array of photosensors arranged in a vertical orientation insome embodiments. To enable stereo image capture, each photosensor 508of first array of photosensors 502 can be in the same path of rotationas a corresponding photosensor 510 in second array of photosensors 504.For instance, corresponding pairs of photosensors 508 and 510 can be inthe same horizontal path of rotation when sensor array 500 is rotatedaround a vertical axis 512, e.g. center axis 218 in FIG. 2. That way,the field of view of first array of photosensors 502 can overlap withthe field of view of the second array of photosensors 504 to capture thesame image of the scene, but from different perspectives, when the fieldof view of sensor array 500 is moved, e.g., by physical rotation, asaforementioned herein with respect to FIGS. 3A-3B.

Gap 506 can be defined by the distance between correspondingphotosensors between first array of photosensors 502 and second array ofphotosensors 504 that are positioned along a movement direction. Forinstance, if sensor array 500 is rotated around a vertical axis 512 sothat its field of view moves horizontally across a scene, gap 506 can bedefined by corresponding photosensors that are positioned along ahorizontal line, e.g., the fourth photosensor from the top of firstarray of photosensors 502 and the fourth photosensor from the top ofsecond array of photosensors 504 as shown in FIG. 5A. That way, thefield of view of those photosensors may overlap when sensor array 500rotates around vertical axis 512 to capture images for stereo imagingpurposes. As discussed herein, gap 506 can be used to determine anoptical baseline for calculating distances to objects in the scene andfor generating images with a perception of depth. Accordingly, gap 506can be any distance suitable for providing an optical baseline that canbe used for triangulation to calculate distance to objects in a scene.For instance, gap 506 can range between 4 to 8 cm, such as approximately6 cm.

It is to be appreciated that gap 506 is not limited to correspondingphotosensors positioned along a horizontal line. As an example, if theaxis of rotation is positioned at a diagonal, e.g., diagonal axis 514,then the gap defining the optical baseline can be defined by thedistance between corresponding photosensors positioned along a linetransverse to diagonal axis 514, such as diagonal rotational line 516.Accordingly, the corresponding photosensors would be the thirdphotosensor from the bottom in first array of photosensors 502 and thethird photosensor from the top in second array of photosensors 502, asshown in FIG. 5A. That way, the field of view of those photosensors mayoverlap when sensor array 500 rotates around diagonal axis 514 tocapture images for stereo imaging purposes. It is to be appreciated thatsensor array 500 can rotate in any direction to perform stereoscopicimaging, and thus be used to define a gap for establishing an opticalbaseline based on the direction of the movement of the field of view,without departing from the spirit and scope of the present disclosure.

In addition to linear arrays, some sensor arrays can have photosensorsarranged as two separate m×n arrays where m and n are greater than 1.For instance, as shown in FIG. 5B, both a first array of photosensors522 and a second array of photosensors 524 can each be formed of a 5×15array of photosensors spaced apart by a distance 526. Having a greaternumber of photosensors in each array as compared to sensor array 500 canallow for each array of photosensors 522 and 524 to have a greater fieldof view than arrays of photosensors 502 and 504 so that fewer images mayneed to be captured to image a scene as sensor array 501 rotates 350°.The gap for determining the optical baseline may not be defined bydistance 526 between arrays 522 and 524, because the field of view of aphotosensor positioned at the right edge of first array of photosensors522 may not correspond with the field of view of a photosensorpositioned at the left edge of second array of photosensors 522 forstereoscopic imaging construction and distance calculation. Rather, thedistance between corresponding photosensors in first array ofphotosensors 522 and second array of photosensors 524 may represent thegap used for calculating distance and creating perception of depth.

For instance, a gap 528 defined by the distance between correspondingphotosensors, e.g., top left photosensor 520 of first array ofphotosensors 522 and top left photosensor 522 of second array ofphotosensors 524, can be used to determine an optical baseline forcalculating distance and creating perception of depth. And, because thefield of view of the two arrays overlap, the length of gap 528 may beshared across all corresponding photosensors. As an example, the lengthof a gap 534 defined by the distance between corresponding photosensors,e.g., bottom right photosensor 536 of first array of photosensors 522and bottom right photosensor 538 of second array of photosensors 524,and all other gaps for corresponding photosensors, can be equal to thelength of gap 528. That way, the field of view of first array ofphotosensors 522 can overlap with the field of view of second array ofphotosensors 522 to capture two offset images of the scene for stereoimaging and distance calculation purposes.

Although FIGS. 5A and 5B illustrate sensor arrays having two array ofphotosensors separated by a region of a sensor array with nophotosensors, embodiments are not limited to such configurations.Rather, embodiments can be implemented in imager systems where thesensor array is configured as a single, two-dimensional array ofphotosensors. For example, with reference to FIG. 5C, sensor array 503can include a single, 20×15 array of photosensors 540. Sensor array 503can be configured to perform stereo imaging and distance calculation byallocating images captured by subsets of photosensors for stereo imagingand distance calculation purposes. Those subsets may be strategicallypositioned so that, as the field of view of sensor array 503 moves(e.g., by rotation), two offset images can be captured.

As an example, array of photosensors 540 can include two subsets ofarrays of photosensors: a first subset of photosensors 542 and a secondsubset of photosensors 544 spaced apart from one another. The dimension,configuration, and operation of first and second subsets of photosensors542 and 544 may correspond to the dimension, configuration, andoperation of first and second arrays of photosensors 522 and 524discussed herein with respect to FIG. 5B. The distance between first andsecond subsets of photosensors 542 and 544 may be populated with otherphotosensors 546 that may not be utilized for stereo imaging anddistance calculation purposes, but for only capturing 2D images of thescene. As can be appreciated herein, some embodiments can utilize imagesensors typically used for 2D image capture in a unique way to performstereo imaging and distance calculation, and thus be more cost effectiveand simpler to design.

Although FIGS. 5A-5C illustrate example sensor arrays having lineararrays, 5×15 arrays, and 20×15 arrays, embodiments are not limited tosuch configurations. It is to be appreciated that embodiments herein canhave any number, size, and arrangement of photosensors suitable forstereoscopic imaging and distance calculation purposes without departingfrom the spirit and scope of the present disclosure.

II. Active Stereoscopic Imager Systems

As discussed herein, stereoscopic imager systems can also be configuredas active stereoscopic imager systems. Active stereoscopic imagersystems can differ from passive stereoscopic imager systems in thatactive stereoscopic imager systems can also emit their own light into afield and detect the emitted light after it has reflected off surface(s)of an object in the field. In some embodiments, active stereoscopicimager systems can also be utilized as LIDAR devices where emitted andreceived, reflected light can be correlated to determine a distance tothe object from which the emitted light was reflected. A large number ofdistance data points can be collected by an active stereoscopic imagersystem and processed to form a three-dimensional point cloudrepresenting the scene in the field of view of the system as captured bythe LIDAR device. A better understanding of an active stereoscopicimager system can be ascertained with reference to FIG. 6.

FIG. 6 is a block diagram of an active stereoscopic system 600,according to some embodiments of the present disclosure. Activestereoscopic system 600 can include a light ranging device 602 and auser interface 615. Light ranging device 602 can include a rangingsystem controller 604, a light transmission (Tx) module 606 and a lightsensing (Rx) module 608. Ranging data can be generated by light rangingdevice 602 by transmitting one or more light pulses 610 from the lighttransmission module 606 to objects in a field of view surrounding lightranging device 602. Reflected portions 612 of the transmitted light arethen detected by light sensing module 608 after some delay time. Basedon the delay time, the distance to the reflecting surface can bedetermined. Other ranging methods can be employed as well, e.g.continuous wave, Doppler, and the like.

Tx module 606 includes an emitter array 614, which can be aone-dimensional or two-dimensional array of emitters, and a Tx opticalsystem 616, which when taken together with emitter array 614 can form alight emission system 638. Tx optical system 616 can include a bulktransmitter optic that is image-space telecentric. In some embodiments,Tx optical system 616 can further include one or more micro-opticstructures that increase the brightness of beams emanating from the bulktransmitter optic and/or for beam shaping, beam steering or the like.Emitter array 614 or the individual emitters can be laser sources. Txmodule 606 can further include an optional processor 618 and memory 620,although in some embodiments these computing resources can beincorporated into ranging system controller 604. In some embodiments, apulse coding technique can be used, e.g., Barker codes and the like. Insuch cases, memory 620 can store pulse-codes that indicate when lightshould be transmitted. In some embodiments, the pulse-codes are storedas a sequence of integers stored in memory.

Light sensing module 608 can be substantially similar in construction tolight sensing module 106 discussed herein with respect to FIG. 1. Thus,details of processor 622, memory 624, sensor array 626, and Rx opticalsystem 628 (when taken together with sensor array 626 can form a lightdetection system 636) can be referenced herein with respect to FIG. 1,and only differences with respect to those components are discussedherein for brevity. For active stereoscopic system 600, sensor array 626can include ranging photosensors. Each ranging photosensor can be aplurality of photodetectors, such as a mini-array of multiplesingle-photon avalanche detectors (SPADs), or a single photon detector(e.g., an APD). In some embodiments, the ranging photosensors of sensorarray 626 can correspond to a particular emitter of emitter array 614,e.g., as a result of a geometrical configuration of light sensing module608 and Tx module 606. For example, in some embodiments, emitter array614 can be arranged along the focal plane of the bulk transmitter opticsuch that each illuminating beam projected from the bulk transmitteroptic into the field ahead of the system is substantially the same sizeand geometry as the field of view of a corresponding receiver channel atany distance from the system beyond an initial threshold distance. Inaddition to the ranging photosensors, sensor array 626 can also includeimaging photosensors. For instance, imaging photosensors such as CCD orCMOS sensors can be positioned and configured to capture images forstereoscopic imaging and distance calculation purposes. Such sensorarrays are discussed further herein with respect to FIG. 8.

In some embodiments, processor 618 can perform signal processing of theraw histograms from the individual photon detectors (or groups ofdetectors) in the array. As an example of signal processing, for eachphoton detector or grouping of photon detectors, memory 624 (e.g., SRAM)can accumulate counts of detected photons over successive time bins, andthese time bins taken together can be used to recreate a time series ofthe reflected light pulse (i.e., a count of photons vs. time). Thistime-series of aggregated photon counts is referred to herein as anintensity histogram (or just histogram). Processor 618 can implementmatched filters and peak detection processing to identify return signalsin time. In addition, Processor 618 can accomplish certain signalprocessing techniques (e.g., by processor 622), such as multi-profilematched filtering to help recover a photon time series that is lesssusceptible to pulse shape distortion that can occur due to SPADsaturation and quenching. In some embodiments, all or parts of suchfiltering can be performed by processor 458, which may be embodied in anFPGA.

In some embodiments, the photon time series output from processor 618are sent to ranging system controller 604 for further processing, e.g.,the data can be encoded by one or more encoders of ranging systemcontroller 604 and then sent as data packets to user interface 615.Ranging system controller 604 can be realized in multiple waysincluding, e.g., by using a programmable logic device such an FPGA, asan ASIC or part of an ASIC, using a processor 630 with memory 632, andsome combination of the above. Ranging system controller 604 cancooperate with a stationary base controller or operate independently ofthe base controller (via pre-programed instructions) to control lightsensing module 608 by sending commands that include start and stop lightdetection and adjust photodetector parameters. Similarly, ranging systemcontroller 604 can control light transmission module 606 by sendingcommands, or relaying commands from the base controller, that includestart and stop light emission controls and controls that can adjustother light-emitter parameters (e.g., pulse codes). In some embodiments,ranging system controller 604 has one or more wired interfaces orconnectors for exchanging data with light sensing module 608 and withlight transmission module 606. In other embodiments, ranging systemcontroller 604 communicates with light sensing module 608 and lighttransmission module 606 over a wireless interconnect such as an opticalcommunication link.

Light ranging device 602 can be used in scanning architectures where Rxmodule 608 and Tx module 606 physically rotates together by way of anelectric motor 634, or the field of view rotates via a mirror device,such as a MEMS device, while Rx module 608 and Tx module 606 arestationary. Thus, electric motor 634 is an optional component in activestereoscopic imager system 600 that can be used to rotate systemcomponents, e.g., Tx module 606 and Rx module 608, as part of a LIDARand stereo image capture architecture. System controller 604 can controlelectric motor 634 and can start rotation, stop rotation and vary therotation speed as needed to implement a scanning system.

Active stereoscopic imager system 600 can interact with one or moreinstantiations of a user interface 615. The different instantiations canvary and can include, but not be limited to, a computer system with amonitor, keyboard, mouse, CPU and memory; a touch-screen in anautomobile or other vehicle; a handheld device with a touch-screen; orany other appropriate user interface. User interface 615 can be local tothe object upon which active stereoscopic imager system 600 is mountedbut can also be a remotely operated system. For example, commands anddata to/from active stereoscopic imager system 600 can be routed througha cellular network (LTE, etc.), a personal area network (Bluetooth,Zigbee, etc.), a local area network (WiFi, IR, etc.), or a wide areanetwork such as the Internet. User interface 615 of hardware andsoftware can present the LIDAR data from the device to the user or to avehicle control unit (not shown) but can also allow a user to controlactive stereoscopic imager system 600 with one or more commands. Examplecommands can include commands that activate or deactivate the activestereoscopic imager system, specify photodetector exposure level, bias,sampling duration and other operational parameters (e.g., emitted pulsepatterns and signal processing), specify light emitters parameters suchas brightness. In addition, commands can allow the user to select themethod for displaying results. The user interface can display activestereoscopic imager system results which can include, e.g., a singleframe snapshot image, a constantly updated video image, and/or a displayof other light measurements for some or all pixels. In some embodiments,user interface 615 can track distances (proximity) of objects from thevehicle, and potentially provide alerts to a driver or provide suchtracking information for analytics of a driver's performance.

In some embodiments, for example where active stereoscopic imager system600 is used for vehicle navigation, user interface 615 can be a part ofa vehicle control unit that receives output from, and otherwisecommunicates with light ranging device 602 and/or user interface 615through a network, such as one of the wired or wireless networksdescribed above. One or more parameters associated with control of avehicle can be modified by the vehicle control unit based on thereceived ranging data. For example, in a fully autonomous vehicle,active stereoscopic imager system 600 can provide a real time 3D imageof the environment surrounding the car to aid in navigation inconjunction with GPS and other data. In other cases, active stereoscopicimager system 600 can be employed as part of an advanceddriver-assistance system (ADAS) or as part of a safety system that,e.g., can provide 3D image data to any number of different systems,e.g., adaptive cruise control, automatic parking, driver drowsinessmonitoring, blind spot monitoring, collision avoidance systems, etc.When user interface 615 is implemented as part of a vehicle controlunit, alerts can be provided to a driver or tracking of a proximity ofan object can be tracked.

FIG. 7 is a simplified diagram illustrating a detailed view of anexemplary active stereoscopic imager system 700 having a widefield-of-view and capable of narrowband imaging, according to someembodiments of the present disclosure. Unlike passive stereoscopicimager systems, active stereoscopic imager system 700 can include both alight detection system 701 and a light emission system 702. Lightemission system 702 provides active illumination of at least a portionof a field in which system 700 is positioned with narrowband light rays704. Light detection system 701 detects the narrowband light emittedfrom the light emission system 702 after it has been reflected byobjects in the field as reflected light rays 706.

A. Light Emission System

In some embodiments, light emission system 702 includes a bulktransmitter optic 718 and a light emitting layer 720 formed of a one- ortwo-dimensional array of light emitters 722. Each light emitter 722 canbe configured to generate discrete beams of narrowband light. In someembodiments, light emitting layer 720 is configured to selectivelyproject the discrete beams of light through bulk transmitter optic 718according to an illumination pattern that matches, in size and geometryacross a range of distances from light emission system 702, the fieldsof view of corresponding receiver channels in micro-optic receiverchannel array 714. Light emitters 722 can be any suitable light emittingdevice, such as a vertical-cavity surface-emitting lasers (VCSELS)integrated on one or more monolithic chip, or any other type of laserdiode. Light emitters 722 can produce cones of narrowband light 724 thatare directed to bulk transmitter optic 718, which can collimate cones oflight 724 and then output the collimated light to distant targets in thefield as emitted light rays 704. In some embodiments, bulk transmitteroptic 718 is image-space telecentric.

B. Light Detection System

Once the light is emitted into the field by light emission system 702,corresponding ranging photosensors in light detection system 701 canreceive the emitted light once they have reflected off of objects in thefield. The received reflected light can be used to identify distances tothe objects in the field. According to some embodiments of the presentdisclosure, light detection system 701 can also include imagingphotosensors for performing stereoscopic imaging, as will be discussedfurther herein.

Light detection system 701 can be representative of light detectionsystem 112 discussed above with respect to FIG. 1 and can include a bulkreceiver optic 708 and a micro-optic receiver (Rx) layer 714. Duringoperation, light rays 706 enter bulk receiver optic 708 from multipledirections and gets focused by bulk receiver optic 708 to form lightcones 710. Micro-optic receiver layer 714 is positioned so thatapertures 726 coincide with the focal plane of bulk receiver optic 708.In some embodiments, micro-optic receiver layer 714 can be aone-dimensional or two-dimensional array of micro-optic receiverchannels 712, where each micro-optic receiver channel 712 is formed of arespective aperture 726, collimating lens 728, and photosensor 716positioned along the same axis in the direction of light flow, e.g.,horizontal from left to right as shown in FIG. 2. Furthermore, eachmicro-optic receiver channel 712 can be configured various ways tomitigate interference from stray light between photosensors, as will bediscussed further herein. During operation, each micro-optic receiverchannel 712 measures light information for a different pixel (i.e.,position in the field).

At the focal point of bulk receiver optic 708, light rays 706 focus andpass through apertures 726 in an aperture layer 711 and into respectivecollimating lenses 728. Each collimating lens 728 collimates thereceived light so that the light rays all enter the optical filter atapproximately the same angle, e.g., parallel to one another. Theaperture and focal length of bulk receiver optic 708 determine the coneangle of respective light rays that come to a focus at aperture 726. Theaperture size and the focal length of collimating lenses 728 determinehow well-collimated the admitted rays can be, which determines hownarrow of a bandpass can be implemented in optical filter 730 to blockunwanted wavelengths of light. Apertures 726 can serve various functionsduring the operation of light detection system 701. For instance,apertures 726 can (1) constrain the pixel FOV so it has tight spatialselectivity despite a large pitch at the photosensor plane, (2) providea small point-like source at the collimating lens's focal plane toachieve tight collimation of rays before passing through the filter,where better collimation results in a tighter band that can pass throughthe filter, and (3) reject stray light.

In some embodiments, photosensors 716 are positioned on a side oppositeof collimating lenses 728 so that light rays 706 first pass throughcollimating lenses 728 and optical filter 730 before exposing onphotosensors 716. Some photosensors 716 can be ranging photosensorsconfigured to receive the emitted light, such as a plurality ofphotodetectors, e.g., a mini-array of multiple single-photon avalanchedetectors (SPADs). An array of mini-arrays of SPADs can be fabricated ona single monolithic chip, thereby simplifying fabrication. In somealternative embodiments, each photosensor 716 can be a singlephotodetector, e.g., a standard photodiode, an avalanche photodiode, aresonant cavity photodiode, or another type of photodetector. Otherphotosensors can be configured as imaging photosensors for stereoscopicimaging purposes, as will be discussed further herein.

C. Sensor Array for Active Stereoscopic Imaging

As discussed above, an active stereoscopic imager system can have asensor array that includes ranging photosensors and imagingphotosensors. The ranging photosensors can correlate with the emittersso that light, e.g., narrowband laser light, emitted by the emitters canbe captured by the ranging photosensors for calculating distances viatime-of-flight measurements and the like. The imaging photosensors maynot be configured to capture emitted light, but can be insteadconfigured to capture offset red-green-blue (RGB) images of the scenefrom ambient light for stereo imaging and distance calculation purposes.In such embodiments, the distance calculated by way of stereo imagingcan be used to augment measurements made by the ranging photosensorsand/or to fill in measurement gaps where ranging photosensors may not beas accurate, such as measurements for very near range (e.g., 0-2 meters)and the like. An example of such a sensor array is discussed herein withrespect to FIG. 8.

FIG. 8 is a simplified illustration of an example sensor array 800 foran active stereoscopic imager system, according to some embodiments ofthe present disclosure. Sensor array 800 can include an array of rangingphotosensors 802 positioned between two arrays of imaging photosensors:a first array of imaging photosensors 804 and a second array of imagingphotosensors 806. Array of ranging photosensors 802 can be the same typeof photosensor than first and second arrays of imaging photosensors 804and 806. For instance, ranging photosensors 802 and imaging photosensors804 and 806 can be SPADs or other avalanche diode sensors configured tomeasure narrowband laser light. However, the light sensed by eitherarray of photosensors can be different. As an example, rangingphotosensors 802 can sense light emitted from the emitter array whereasimaging photosensors 804 and 806 are configured to sense ambient light.To enable imaging photosensors 804 and 806 to measure light, the opticalfilter in the receiver channels for corresponding photosensors 804 and806 may be configured to allow certain wavelengths of visible light.

In some additional and alternative embodiments, array of rangingphotosensors 802 can be a different type of photosensor than first andsecond arrays of imaging photosensors 804 and 806. As an example, arrayof ranging photosensors 802 can be SPADs or other avalanche diodesensors, whereas first and second arrays of imaging photosensors 804 and806 can be CCD, CMOS, and other similar types of photosensors configuredto measure a wider band of light, e.g., visible RGB ambient light.

Because of its correlation with the emitter array, array of rangingphotosensors 802 can have an array size and photosensor pitch suitablefor receiving light emitted by the emitter array, while first and secondarrays of imaging photosensors 804 and 806 can have an array size andphotosensor pitch different from both the emitter array and the array ofranging photosensors 802. For instance, as shown in FIG. 8, array ofranging photosensors 802 can have a diagonally staggered arrangement ina downward translation to the right, and first and second arrays ofimaging photosensors 804 and 806 can each have a two-dimensional,rectangular m×n arrangement of imaging photosensors that are moredensely packed than array of ranging photosensors 802. In suchembodiments, each ranging photosensor, e.g., ranging photosensor 808,can correspond to a set of imaging photosensors, e.g., set of imagingphotosensors 810, that are in the same row as the corresponding rangingphotosensor 808, so that as sensor array 800 rotates around to image thescene, set of imaging photosensors 810 can capture ambient lightexisting at the same field of view as ranging photosensor 808 when eachphotosensor of set of imaging photosensors 810 is correspondinglypositioned. Thus, light captured by sensor array 800 can be used toconstruct a topographical map of the scene, as well as an RGB image ofthe scene that is highly correlated with the topographical map of thescene.

In addition to device type, pitch, and arrangement differences, array ofranging photosensors 802 can be constructed of photosensors that have adifferent size than first and second arrays of imaging photosensors 804and 806. As an example, a ranging photosensors can have largerdimensions than an image photosensor. That way, the ranging photosensorcan be better equipped to measure emitted light that is reflected off ofobjects in the scene, such as by being constructed of a SPAD.

According to some embodiments of the present disclosure, subsets ofimaging photosensors from first and second arrays of imagingphotosensors 804 and 806 can be allocated to provide data that can beused for stereo imaging purposes, such as the stereo imaging purposesdiscussed herein with respect to FIGS. 3A-3C and 5A-5B. For example, afirst subset of imaging photosensors 812 from first array ofphotosensors 804 and a second subset of imaging photosensors 814 fromsecond array of photosensors 804 can be used for stereoscopic imagingpurposes. In some embodiments, first and second subsets of imagingphotosensors 812 and 814 can each include a portion of the entire arrayof imaging photosensors in the first and second arrays of photosensors804 and 806, as shown in FIG. 8. This may be because those imagingphotosensors (e.g. the top to rows of first array of imagingphotosensors 804 and the bottom two rows of the second array of imagingphotosensors 806) that are not part of the subsets of imagingphotosensors may not have corresponding photosensors in the other array804/806 that are positioned in the same path of rotation, e.g., the samehorizontal line in instances where sensor array 800 rotates around avertical axis. Thus, those imaging photosensors may not have acorresponding offset image for stereo imaging purposes.

As can be appreciated with reference to FIG. 8, subset of imagingphotosensors 812 can be spaced apart from subset of imaging photosensors814, and thus be corresponding subsets of imaging photosensors forgenerating two offset images of a scene for calculating distance andcreating perception of depth. Subsets of imaging photosensors 812 and814 can have similar features and arrangements as arrays of photosensors622 and 624 and subsets of photosensors 642 and 644 discussed hereinwith respect to FIGS. 5B and 5C.

In some embodiments, the accuracy of distances measured by array ofranging photosensors 802 can be augmented and/or superseded by distancemeasurements calculated from analysis of the two offset stereo imagescaptured by subsets of imaging photosensors 812 and 814. As an example,accuracy of measurements at very close distances, e.g., 0-2 meters,measured by array of ranging photosensors 802 may be lower than itsaccuracy of measurements at farther distances, e.g., 2+ meters. Thus,those distances calculated by array of ranging photosensors 802 can beaugmented by distances calculated by analysis of the stereo image pair.One way of augmenting the measurements can be by averaging the twodistances together. That way, a more accurate measurement of thedistance can be ascertained. In some additional and alternativeembodiments, those distances calculated by array of ranging photosensors802 can be replaced by distances calculated by analysis of the stereoimage pair. In such instances, those distances calculated by array ofranging photosensors 802 can simply be ignored. Either form ofaugmentation can be performed by default, or in response to adetermination that the difference between the first and second distancesis greater than a threshold.

III. Method of Operating Stereoscopic Imager Systems

FIG. 9 is a block diagram of an example method 900 of operating astereoscopic imager system to perform stereoscopic imaging using lightdetections systems, according to some embodiments of the presentdisclosure. At step 902, a field of view of a sensor array can be movedto capture images of a scene. For instance, the sensor array can berotated around a center axis, as discussed herein with respect to FIGS.3A-3C, or one or more mirrors, such as a MEMS device, can be moved toalter the field of view of the sensor array. The sensor array caninclude a first photosensor and a second photosensor spaced apart fromthe first photosensor to capture the images of the scene. As an example,the first photosensors can be any photosensor from first array/subset ofphotosensors 502, 522, 542, and 804, and the second photosensor can beany photosensor from second array/subset of photosensors 504, 524, 544,and 806 corresponding to the first photosensor, as discussed herein withrespect to FIGS. 5A-5C and 8. In some embodiments, the first photosensorand the second photosensor can be separated by a gap, e.g., gap 506 or528 discussed herein with respect to FIGS. 5A-5B.

At step 904, a first image of the object in the scene can be capturedfrom a first perspective at a first instance of time as the field ofview of the sensor array moves. As an example, as the sensor arrayrotates around the center axis, a first array of photosensors includingthe first photosensor can capture an image of the object in the scene,as discussed herein with respect to FIG. 3A. Then, at step 906, as thesensor array continues to move, but before making a full 360° rotationafter the first image is captured, a second image of the object in thescene can be captured from a second perspective at a second instance oftime. For instance, as the sensor array rotates around the center axis,a second array of photosensors including the second photosensor cancapture an image of the object in the scene, as discussed herein withrespect to FIG. 3B. The dimensions of the field of view for the firstand second arrays of photosensors can be the same so that the two imagescan form a stereo image pair.

At step 910, the first image and the second image can be analyzed tocalculate a first distance to the object in the scene using an opticalbaseline based on the gap between the first photosensor and the secondphotosensor. In some embodiments, a processor, e.g., processor 122 ofsystem controller 104 discussed herein with respect to FIG. 1, of thestereoscopic imager system 100 can receive the first and second images,compare shared features of the object apparent in the first and secondimages, and use the distance between those shared features to identify adistance to the object using triangulation, as discussed herein withrespect to FIG. 5. Accordingly, embodiments of the present disclosurecan use a two-dimensional sensor array to capture distance to objects inthe scene.

As discussed herein with respect to FIGS. 7 and 8, the stereoscopicimager system can be an active stereoscopic imager system that can alsoperform ranging using an emitter array and ranging photosensors formeasuring the light emitted by the emitter array after it has reflectedoff the object in the scene. Accordingly, method 900 can optionallyinclude step 912, where a second distance to the object can be measured.The second distance can be measured by a ranging photosensor thatmeasures light emitted by a corresponding light emitter once the lighthas reflected off of the object in the scene. Time-of-flightmeasurements and the like can be used to calculate the distance to theobject.

Thereafter, at step 914, a final distance to the object can bedetermined based on the first distance and the second distance. As anexample, the first distance can be used to augment the accuracy of thesecond distance, especially for very close range measurements, asdiscussed herein with respect to FIGS. 7 and 8. In some other instances,the first distance can replace the second distance. Either form ofaugmentation can be performed by default, or in response to adetermination that the difference between the first and second distancesis greater than a threshold.

IV. Mitigating Receiver Channel Cross-Talk

As can be appreciated by disclosures herein, channels in the micro-opticreceiver and are positioned very close to one another, often timeswithin microns of one another. This small spacing between each channelcan invite the opportunity for problems to arise. For instance, lightpropagating through bulk imaging optic can occasionally cause straylight to bleed into neighboring channels, thereby resulting ininaccurate readings of reflected light for each pixel in the field.Ideally, no stray light should be received by any channel, as shown inFIG. 10A.

FIG. 10A is a simplified cross-sectional view diagram of part of a lightdetection system 1000 where there is no cross-talk between channels.During operation, perpendicular light rays 1002 and chief ray 1004 enterthe bulk imaging optic 1006 and produce light cone 1008. Light rays 1002and 1004 enter an aperture of aperture layer 1010 and enter collimatinglens 1011. Collimating lens 1011 accepts a limited range of incidentlight angles. For example, collimating lens 1011 can accept light raysat incident angles between +25 to −25 degrees relative to theperpendicular. FIG. 10A shows light cone 1008 with incident anglesbetween +25 to −25 degrees. The chief ray 1004 is the light ray thatpasses through the center of the aperture. In this example, the chiefray 1004 has an incident angle of 0 degrees on the collimating lens1011.

FIG. 10B is a simplified cross-sectional view diagram of part of a lightdetection system 1001 where there is cross-talk between channels. Inthis case, during operation, oblique light rays 1012 and chief ray 1014enter bulk receiver optic 1016 and later enter collimating lens 1021. Inthis example, collimating lens 1021 belongs to a micro-optic channelthat corresponds to a photosensor further from the center of the image.In this example, chief ray 1014 has an incident angle of −12 degrees andthe cone of focused light has incident angles between +12 degrees to −35degrees. Collimating lens 1021 rejects some of the light rays because itonly accepts light with incident angles between +25 to −25 degrees.Additionally, the rays that are outside of the collimating lensacceptance cone can travel to other optical surfaces and become straylight. Thus, a non-telecentric bulk imaging optic will deliversignificantly fewer signal photons to the photodetector, whilepotentially polluting other channels with errant light rays 1022. Atelecentric bulk imaging optic, on the other hand, will produce lightcones with incident angles approximately between +25 to −25 degrees andchief rays with incident angles on the collimating lens of approximately0 degrees, regardless of the angle of the oblique rays 1012 and chiefray 1014. A telecentric bulk imaging optic has similar benefits for thetransmitter when the lasers are telecentric (their chief rays are allparallel) as is the case for VCSELS or a side emitter diode laser bar.

In some embodiments, the light detection system of a light sensingmodule uses an input image-space telecentric bulk imaging optic. In someother embodiments, for example where cost or increased field of view ismore important than performance, the light detection system may use amore standard input bulk imaging optic such as a bi-convex lens. For anygiven input field into an image-space telecentric lens, the resultingchief rays are parallel to the optical axis, and the image-side raycones all span approximately the same set of angles. This allowsmicro-optic channels far from the optical axis in the light detectionsystem to achieve similar performance to the on-axis micro-opticchannel. The light detection system does not need perfect image spacetelecentricity for this to work, but the closer to perfecttelecentricity the better. For a micro-optic receiver optical layer lensthat can only accept +/−25 degree light, the preference is that theinput bulk imaging optic produce image-side rays that are no greaterthan 25 degrees in angle for every point on the focal plane.

According to some embodiments of the present disclosure, the design ofeach channel of the micro-optic receiver channel array can bespecifically configured to have features that minimize the intrusion ofstray light onto a respective photodetector, thereby reducing oreliminating any detrimental effects caused by the occurrence of straylight. FIG. 11 is a simplified cross-sectional diagram of an examplemicro-optic receiver channel structure 1100, also called a micro-opticreceiver channel in discussions herein. Receiver channel 1100 can berepresentative of micro-optic receiver channels 712 shown in FIG. 7, andserves to accept an input cone of light containing a wide range ofwavelengths, filters out all but a narrow band of those wavelengthscentered at the operating wavelength, and allows photosensor 1171 todetect only or substantially only photons within the aforementionednarrow band of wavelengths. According to some embodiments of the presentdisclosure, micro-optic receiver channel structures, such as receiverchannel 1100, can include the following layers:

-   -   An input aperture layer 1140 including an optically transparent        aperture 1144 and optically non-transparent stop region 1146        configured to define a narrow field of view when placed at the        focal plane of an imaging optic such as bulk receiver optic 708        (shown in FIG. 7; not shown in FIG. 11). Aperture layer 1140 is        configured to receive the input marginal ray lines 1133. The        term “optically transparent” herein refers to as allowing most        or all light to pass through. Light herein refers to spectrum of        light in the near-ultraviolet, visible, and near-infrared range        (e.g. 300 nm to 5000 nm). Optically non-transparent herein        refers to as allowing little to no light to pass through, but        rather absorbing or reflecting the light. Aperture layer 1140        can include optically transparent apertures separated from each        other by optically non-transparent stop regions. The apertures        and stop regions can be built upon a single monolithic piece        such as an optically transparent substrate. Aperture layer 1140        can optionally include a one-dimensional or two-dimensional        array of apertures 1144.    -   An optical lens layer 1150 including a collimating lens 1151        characterized by a focal length, offset from the plane of        aperture 1144 and stop region 1146 by the focal length, aligned        axially with aperture 1144, and configured to collimate photons        passed by the aperture such that they are traveling        approximately parallel to the axis of collimating lens 1151        which is aligned with the optical axis of receiver channel 1100.        Optical lens layer 1150 may optionally include apertures,        optically non-transparent regions and tube structures to reduce        cross talk.    -   An optical filter layer 1160 including an optical filter 1161,        typically a Bragg reflector type filter, adjacent to collimating        lens 1151 and opposite of aperture 1144. Optical filter layer        1160 can be configured to pass normally incident photons at a        specific operating wavelength and passband. Optical filter layer        1160 may contain any number of optical filters 1161. Optical        filter layer 1160 may optionally include apertures, optically        non-transparent regions and tube structures to reduce cross        talk.    -   A photosensor layer 1170 including a photosensor 1171 adjacent        to optical filter layer 1160 and configured to detect photons        incident on photosensor 1171. Photosensor 1171 herein refers to        a single photodetector capable of detecting photons, e.g., an        avalanche photodiode, a SPAD (Single Photon Avalanche Detector),        RCP (Resonant Cavity Photo-diodes), and the like, or several        photodetectors, such as an array of SPADs, cooperating together        to act as a single photosensor, often with higher dynamic range,        lower dark count rate, or other beneficial properties as        compared to a single large photon detection area. Photosensor        1171 can also refer to visible light photosensors, such as CCD        and CMOS sensors for capturing stereo images. Each photodetector        can be an active area that is capable of sensing photons, i.e.,        light. Photosensor layer 1170 refers to a layer made of        photodetector(s) and contains optional structures to improve        detection efficiency and reduce cross talk with neighboring        receiver structures. Photosensor layer 1170 may optionally        include diffusers, converging lenses, apertures, optically        non-transparent tube spacer structures, optically        non-transparent conical spacer structures, etc.

Stray light may be caused by roughness of optical surfaces,imperfections in transparent media, back reflections, and the like, andmay be generated at many features within the receiver channel 1100 orexternal to receiver channel 1100. The stray light may be directed:through the filter region 1161 along a path non-parallel to the opticalaxis of collimating lens 1151; reflecting between aperture 1144 andcollimating lens 1151; and generally taking any other path or trajectorypossibly containing many reflections and refractions. If multiplereceiver channels are arrayed adjacent to one another, this stray lightin one receiver channel may be absorbed by a photosensor in anotherchannel, thereby contaminating the timing, phase, or other informationinherent to photons. Accordingly, receiver channel 1100 may featureseveral structures to reduce crosstalk between receiver channels.

V. Implementation of Stereoscopic Imager Systems

FIG. 12A is a top-down view of a simplified diagram of an examplestereoscopic imager system 1200 implemented for a vehicle 1205, such asa car, and capable of continuous 360 degree scanning, according to someembodiments of the present disclosure. The output beam(s) of one or morelight sources (such as infrared or near-infrared pulsed IR lasers, notshown) located in stereoscopic imager system 1200, can be scanned, e.g.,rotated, to illuminate a continuous scene 1220 around the vehicle. Insome embodiments, the scanning, represented by rotation arrow 1215, canbe implemented by any suitable mechanical means, e.g., by mounting thelight emitters to a rotating column or platform, or any other mechanicalmeans, such as through the use of galvanometers or chip-based steeringtechniques. During operation, objects around vehicle 1205 in anydirection and within the view of stereoscopic imager system 1200 canreflect portions of light pulses 1211 that are emitted from atransmitting module 1208 in stereoscopic imager system 1200. One or morereflected portions 1217 of light pulses 1211 then travel back tostereoscopic imager system 1200 and can be detected by its sensingmodule 1209. Additionally, ambient light from scene 1220 can be capturedto generate two offset images for distance calculation and depthperception purposes. In some instances, sensing module 1209 can bedisposed in the same housing as transmitting module 1208.

Although FIG. 12A illustrates solid state stereoscopic imager systemsmounted on a roof of a vehicle 1205, embodiments are not limited to suchconfigurations. Other embodiments can have solid state stereoscopicimager systems mounted on other regions of a vehicle. For instance,stereoscopic imager systems can be mounted at the corners of a vehicle,as shown in FIG. 12B. FIG. 12B illustrates an implementation 1201 wheresolid state stereoscopic imager systems 1204 a-d are implemented at theouter regions of a road vehicle, such as a car, according to someembodiments of the present disclosure. In this implementation, eachstereoscopic imager system 1204 a-d can be a spinning stereoscopicimager system that can measure distances around the full 360 degrees.However, since at least some of those measurements will be measured withrespect to vehicle 1205, those measurements can be ignored. Thus, eachstereoscopic imager system 1205 a-d can utilize a subset of themeasurements from 360 degree scanning, e.g., only the angles coveringregions 1219 a-d that do not capture vehicle 1205 are utilized.

Although the present disclosure has been described with respect tospecific embodiments, it will be appreciated that the present disclosureis intended to cover all modifications and equivalents within the scopeof the following claims.

What is claimed is:
 1. A stereoscopic imager system, comprising: asensor array comprising: a plurality of ranging photosensors that detectlight emitted from an emitter array once it has reflected off of anobject in a scene; a first plurality of imaging photosensors positionedat a first side of the ranging photosensors; and a second plurality ofimaging photosensors positioned at a second side of the rangingphotosensors opposite from the first side, the first plurality ofimaging photosensors and the second plurality of imaging photosensorsdetect ambient light in the scene and are spaced apart by a gap; amoving component coupled to the sensor array and operable to move thesensor array between a first position and a second position within afull rotational image capturing cycle; and a system controller coupledto the sensor array and the moving component, the system controllerconfigured to: determine a first distance to an object in the sceneusing the plurality of ranging photosensors by way of time-of-flightcalculations; capture a first image of the scene at the first positionwith the first plurality of imaging photosensors and a second image ofthe scene at the second position with the second plurality of imagingphotosensors; calculate a second distance to the object based on thefirst image and the second image and an optical baseline determined bythe gap.
 2. The stereoscopic imager system of claim 1, wherein theplurality of ranging photosensors are organized in a diagonallystaggered arrangement, and the first and second pluralities of imagingphotosensors are each organized in a rectangular arrangement.
 3. Thestereoscopic imager system of claim 2, wherein at least some of thefirst plurality of imaging photosensors and at least some of theplurality of second imaging photosensors are positioned along the samehorizontal line.
 4. The stereoscopic imager system of claim 1, whereinthe moving component is an electric motor that rotates the sensor arrayaround a center axis.
 5. The stereoscopic imager system of claim 1,wherein the moving component is a micro-electrical mechanical system(MEMS) device that reflects light to move the field of view.
 6. Thestereoscopic imager system of claim 1, wherein the system controller isfurther configured to calculate a final distance to the object based onthe first distance and the second distance.
 7. A stereoscopic imagersystem, comprising: a sensor array comprising a first plurality ofphotosensors and a second plurality of photosensors spaced apart fromthe first plurality of photosensors by a gap, the first plurality ofphotosensors and the second plurality of photosensors being configuredto detect ambient light in a scene; a moving component coupled to thesensor array and operable to move the sensor array between a firstposition and a second position within a full rotational image capturingcycle; a system controller coupled to the sensor array and the movingcomponent, the system controller configured to: move a field of view ofa sensor array by instructing the moving component to capture a firstimage of an object in the scene with the first plurality of photosensorsfrom a first perspective at the first position, and to capture a secondimage of the scene of the object in the scene with the second pluralityof photosensors from a second perspective at the second position; andcalculate, based on the first image and the second image, a distance tothe object using an optical baseline defined by the gap.
 8. Thestereoscopic imager system of claim 7, further comprising a lightdetection system, the light detection system comprising: a bulk receiveroptic configured to receive light rays originating from a field externalto the stereoscopic imager system; and an optical assembly having aplurality of micro-optic receiver channels defining a plurality ofdiscrete, non-overlapping fields of view in the field, the opticalassembly comprising: an aperture layer having a plurality of discreteapertures arranged along a focal plane of the bulk receiver optic, thefirst imaging photosensor and the second imaging photosensor beingdisposed behind the aperture layer; and a non-uniform optical filterlayer configured to allow different micro-optic channels to measuredifferent ranges of wavelengths.
 9. The stereoscopic imager system ofclaim 1, wherein the moving component is an electric motor that rotatesthe sensor array around a center axis.
 10. The stereoscopic imagersystem of claim 1, wherein the moving component is a micro-electricalmechanical system (MEMS) device that reflects light to move the field ofview.
 11. A method of distance measurement, comprising: moving a fieldof view of a sensor array including a first imaging photosensor and asecond imaging photosensor spaced apart from the first imagingphotosensor by a gap; capturing a first image of an object in a scenewith the first imaging photosensor from a first perspective at a firstinstance of time as the field of view moves; capturing a second image ofthe scene of the object in the scene with the second imaging photosensorfrom a second perspective at a second instance of time as the field ofview moves; and calculating, based on the first image and the secondimage, a first distance to the object using an optical baseline definedby the gap.
 12. The method of claim 11, wherein moving the field of viewcomprises rotating the sensor array around a center axis.
 13. The methodof claim 11, wherein moving the field of view comprises reflecting lightto move the field of view while the sensor array is stationary.
 14. Themethod of claim 11, further comprising: comparing shared features of theobject captured in the first image and the second image; and using theresults from the comparison to calculate the first distance to theobject.
 15. The method of claim 11, wherein the first imagingphotosensor is included in a first array of imaging photosensors, andthe second imaging photosensor is included in a second array of imagingphotosensors.
 16. The method of claim 15, wherein the first array ofimaging photosensors and the second array of imaging photosensors areeach two-dimensional arrays of imaging photosensors.
 17. The method ofclaim 15, wherein the sensor array is formed of a two-dimensional arrayof imaging photosensors, and the first array of imaging photosensors andthe second array of imaging photosensors are each a subset of thetwo-dimensional array of imaging photosensors.
 18. The method of claim11, further comprising: measuring a second distance to the object usinga ranging photosensor; and determining a final distance to the objectbased on the first distance and the second distance.
 19. The method ofclaim 18 wherein the ranging photosensor is in a two-dimensional arrayof ranging photosensors and the first and second arrays of imagingphotosensors are located on opposing sides of the array of rangingphotosensors.
 20. The method of claim 19 wherein array of rangingphotosensors is organized in a diagonally staggered arrangement and thefirst and second arrays of imaging photosensors are organized in arectangular arrangement.